The present embodiments relate to registering a three-dimensional image dataset of a target region of a patient with a two-dimensional x-ray image of the target region recorded in an acquisition geometry.
In many problematic medical situations, whether it be in diagnostics or when performing minimally invasive interventions, for example, both three-dimensional image data in the form of an image dataset (e.g., a computed tomography or magnetic resonance dataset) and two-dimensional fluoroscopic images (e.g., typically x-ray images) are available for the target region of interest of the patient. In such situations, it is desirable to be able to carry out a joint evaluation of both the three-dimensional image dataset and the two-dimensional x-ray image of the target region (e.g., in the form of a fusion image or by integrating features of one image into the respective other image). The coordinate systems of the three-dimensional image dataset and the two-dimensional x-ray image are to be correlated with one another. This process is usually referred to as registration (e.g., in the present case as 2D/3D registration).
The 2D/3D registration is particularly important in the context of image-guided medical interventions. In such cases, use is often made of x-ray devices having a C-arm on which an x-ray source and an x-ray detector are arranged opposite one another. The real-time monitoring of medical interventions by x-rays may be performed as fluoroscopy, which is why the x-ray images may also be referred to as fluoroscopic images. In the case of real-time guidance, preoperatively acquired three-dimensional image datasets (e.g., CT, magnetic resonance, etc.) may be superimposed as overlays onto the two-dimensional fluoroscopic x-ray images. The accuracy of the overlaying is important in terms of clinical applicability.
It is known in the prior art to perform a 2D/3D registration at the start of the intervention or the fluoroscopic monitoring in order to provide that the original accuracy is maintained. It may, however, happen during the medical intervention that the 2D/3D registration is rendered invalid due to movements of the patient (e.g., movements of the target region). In other words, misregistrations may occur. In order to eliminate this problem, it is known that the person carrying out the intervention may initiate a new registration procedure manually if the inaccurate overlay becomes clearly visible and thus is already able to affect the interventional procedure. In that event, the most recently acquired x-ray image is used to re-establish a 2D/3D registration. It is disadvantageous in this case that the person performing the intervention is interrupted during the interventional procedure as a result of the motion correction.
A different approach to keeping the 2D/3D registration updated provides that the patient, or at least the target region, is automatically tracked during the x-ray monitoring. A motion correction is thus enabled to be performed “on the fly.” If the motion development is considered over time, it is also possible to predict the movement for future x-ray images. However, a valid, accurate 2D/3D registration at the outset is also to be provided for these tracking-based approaches in order to obtain correspondences for estimating the three-dimensional motion. However, the error occurring in the motion estimation is disadvantageously recognizable in the fusion or overlay images, since the tracking errors accumulate and 2D/3D correspondences are lost due to the movement.
Stated in general terms, various central factors of the 2D/3D registration may be identified. To be cited initially in this context as a first important factor is the similarity measure (e.g., comparison measure) used, which describes the accuracy of the 2D/3D registration. Typically, the image intensities (e.g., grayscale value) and gradients are used. The method utilized in the main for 2D/3D registration according to the prior art is the determination of a digitally reconstructed radiograph (DRR). This is a simulated x-ray image for the three-dimensional image dataset, which is therefore determined by forward projection (cf., for example, the article by A. Kubias et al. titled “Extended global optimization strategy for rigid 2D/3D image registration,” in: CAIP, LINCS, Volume 4673, Springer (2007), pp. 759-767). Comparison measures based on the intensity values of the DRR image and the x-ray image may be used as a similarity measure (e.g., the sum of absolute differences (SAD), the sum of squared differences (SSD), and the normalized cross-correlation (NCC)). Examples of gradient-based comparison measures are the gradient cross-correlation (cf. W. Wein et al. titled “2D/3D registration based on volume gradients,” in: Medical Imaging SPIE (2005), pp. 144-150), and the normalized gradient field (NGF) (cf. E. Haber and J. Modersitzki, “Intensity gradient based registration and fusion of multi-modal images,” MICCAI 2006). These measures are usually regarded as a more robust alternative to intensity-based comparison measures.
Also relevant, for example, with regard to movements occurring in the target region, is the applied motion model. The image registration is generally known as an optimization method in which motion is applied to the moving image in order to bring the moving image into correspondence with the stationary image. In this case, the motion may be applied for rigid structures as a rigid movement and articulated (e.g., partially rigid) movement, but also as an elastic movement in which, as a consequence, deformations may also occur. From the viewpoint of the dimensionality, either three-dimensional motion is applied to the three-dimensional image dataset or two-dimensional motion is applied to the two-dimensional x-ray image (e.g., projected two-dimensional x-ray image; DDR). For examples, in this regard, reference is made to the review article by R. Liao et al. titled “A review of recent advances in registration techniques applied to minimally invasive therapy”, in: IEEE Transactions on Multimedia, 15 (5), 2013, pp. 983-1000.
The optimization strategy may be cited as a third important factor. Based on the selected comparison measure (e.g., similarity measure) and the motion model, a numerical optimization method that uses motion parameters of the motion model as parameters to be optimized is performed in order to maximize the similarity or in order to minimize the difference. In this case, gradient-based optimization methods may be applied with respect to the comparison measure. However, gradient-free optimization methods may also be employed. In the already cited article “Extended global optimization strategy for rigid 2D/3D image registration,” for example, A. Kubias et al. propose using an extended global optimization strategy, where an adaptive random search is used at different coarse resolution levels and a local optimizer is applied to a higher resolution level in order to refine the registration. In the tracking-based approaches, the optimization is often based on mathematical relationships between tracked features and the three-dimensional rigid motion model.
In spite of all of the known approaches, the 2D/3D registration, which, for example, is intended to be performed in real time, still continues to represent a challenge. A category of known methods attempts to enable the three-dimensional motion by comparing the projection of the three-dimensional image dataset with the two-dimensional x-ray image (cf., the already cited publications by A. Kubias et al., W. Wein et al., and E. Haber et al.). However, the optimization is rendered more difficult in this case by the loss of the depth information, which occurs as a result of the projection. Another category of known methods uses the back-projection of the x-ray image and compares this with the three-dimensional image dataset, though these approaches require a plurality of two-dimensional x-ray images acquired from different projection directions (e.g., using different acquisition geometries), which is often not the case in medical interventions.
The tracking-based approaches have the advantage that the three-dimensional motion is determined by two-dimensional tracking. However, the tracking-based approaches require a high-quality initialization as well as a reinitialization in order to establish original 2D/3D correspondences or to reestablish the correspondences. DE 10 2013 214 479 A1 proposes a method for tracking a 2D/3D registration during movement in which contours are tracked in sequentially acquired projection images. A target plane may be determined for a given contour point in the two-dimensional x-ray image and an associated starting point in the three-dimensional image dataset. The target plane, the contour point displaced due to the movement, the starting point displaced due to the movement, and the focus (e.g., ultimately, the radiation source in the acquisition geometry) are located in the target plane. A relationship that connects the observable degrees of freedom of movement perpendicular to the course of the contour in the three-dimensional image dataset with general three-dimensional descriptions of the movement (e.g., motion parameters of a motion model) may be established. This equation system may be underdetermined in the case of a single contour point. However, since a plurality of contour points are considered here, an equation system that may be solved in order to determine the motion parameters and consequently to update the motion information results. This process exploits the fact that the contour may be associated with a rigid object that may consequently be mapped by common motion parameters describing a rigid movement. However, the problems of accumulating tracking errors also occur in this case, as does the potential loss of 2D/3D correspondences. With regard to this approach, reference is made to the article by J. Wang et al., titled “Gradient-Based Differential Approach for 3-D Motion Compensation in Interventional 2-D/3-D Image Fusion,” Proceedings of the 2014 International Conference on 3D Vision, pp. 293-300.
In order to improve this concept, and to permit a robust and accurate registration also starting from less exact 2D/3D correspondences and/or, for example, over a longer series of acquired x-ray images, it is proposed in DE 10 2015 208 929 A1 that after an initial transformation has been specified as a test transformation that is to be optimized, at least one rigid reference structure visible in the x-ray image is selected with an associated contour from anatomical structures contained in the image dataset determined in an evaluation. A two-dimensional gradient x-ray image and a three-dimensional gradient dataset of the image dataset are determined. After this, at least one two-dimensional gradient comparison image forward-projected in the acquisition geometry using the test transformation is determined from the gradient dataset of the x-ray image. An environment that best corresponds to a local environment of the contour point based on a comparison measure and extends around a comparison point in the gradient x-ray image is found for a plurality of contour points on the three-dimensional contour of the at least one selected reference structure in the acquisition geometry of the x-ray image in the gradient comparison image from environments extending around test points in the gradient x-ray image. Local two-dimensional displacement information is determined by comparing the contour points with the associated comparison points. Motion parameters of a three-dimensional motion model that describe a movement of the target region between the acquisition of the image dataset and the x-ray image are determined from the local two-dimensional displacement information and a registration transformation describing the registration by correcting the test transformation based on the motion parameters.
In the method disclosed therein, the registration process is therefore based on specific reference structures (e.g., hence on a sparse mapping), instead of using the entire image. The spatial image gradient is used in the comparison instead of the image intensity, since strong gradients contain the most structural information. The search for comparison points corresponding to starting points is conducted locally in a two-dimensional manner between the gradient x-ray image and the gradient comparison image, since the actually observable movement was merely projected and consequently may be different locally within the projection. Ultimately, however, with use being made of the rigid properties of the reference structures, a motion model that contains both the observable and the non-observable movement is used.
The method disclosed therein therefore includes the following steps. First, an initialization, in which, for example, an only roughly required initial registration is specified as an initial transformation, takes place. Next, reference structures are selected (e.g., contours in the three-dimensional image dataset that are to be tracked). Comparison points associated with the contour points are found in the two-dimensional x-ray image by a gradient-correlated matching. After this, the motion estimation may be performed (e.g., based on a point-to-plane correspondence), as was proposed in DE 10 2013 214 479 A1 and may be refined further in accordance with DE 10 2015 208 929 A1.
An aspect common to the approaches described in DE 10 2013 214 479 A1 and DE 10 2015 208 929 A1 is that usually a dense set of contour points or starting points is used in order to represent the surface/contour of the structure of interest. This is not computationally efficient for a real-time application such as is desired to be realized. Moreover, a volume analysis (e.g., 3D Canny) is necessary to determine the three-dimensional contour points or starting points. For this purpose, large amounts of both computational resources and memory resources are to be provided.